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Mathematics for 
Machine Learning 
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@ 
GIKI - FES 


¢In the last lecture we saw the Magic of Learning 
¢In organisms. 
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¢We now look at the “Magic” of Learning 
¢In algorithms & objects... 
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The Magic of Learning! (In 
Machines...) 


¢How do you think machines learn? 


The Magic of Learning! (In 
Machines...) 


¢How do you think machines learn? 
¢Mostly a/gorithmically! 


i: 5 lee 
ft 


Why Did the Robot Cross the Road? 
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| 
Why Did the Rob 


At higher-level, 
algorithms mostly 
logical: 


If bike CROSS NORMAL 
If motorbike CROSS 
QUICKLY 
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But how can algorithms teach a 
machine? 


But how can algorithms teach a 
machine? 


.. And like neurons/organisms, can we 
achieve something COMPLEX using 


REPETITONS of rather SIMPLE UNITS 
with rather SIMPLE RULES? 


To Murree or Not to Murree... 


To Murree or Not to Murree... 
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To Murree or Not to Murree... 


Let’s go 
to 
Murree 
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To Murree or Not to Murree... 


Let’s go 
to 
Murree 


ES691 - Mathematics for Machine Learning / Dr. Naveed R. Butt @ GIKI - FES 


To Murree or Not to Murree... 


Let’s go 
to 
Murree 
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To Murree or Not to Murree... 


Let’s go 
to 
Murree 
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Apply 
for leave? 


To Murree or Not to Murree... 


If input > 0.3 
apply for 
leave 


Else 
Activation ignore 
Let’s go 
to 
Murree Apply 


for leave? 
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To Murree or Not to Murree... 


If input > 0.3 
apply for 
leave 


Else 
Activation ignore 
Let’s go 
to 
Murree Apply 


for leave? 
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To Murree or Not to Murree... 


If input > 0.3 
apply for 
leave 


Else 
Activation ignore 
Let’s go 
to 
Murree Apply 


for leave? 
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To Murree or Not to Murree... 


If input > 0.3 
apply for 
leave 


Else 
Activation ignore 
Let’s go 
to 
Murree Apply 


for leave? 
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To Murree or Not to Murree... 


If input > 0.3 
apply for 
| leave 
Else 
Activation ignore 


Let’s go 
to 
Murree Apply 

for leave? 


I’ve 
applied 
fora 
leave 
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To Murree or Not to Murree... 


If input > 0.3 
apply for 
| leave 
Else 
Activation ignore 


Let’s go 
to 
Murree Apply 

for leave? 


I’ve 
applied 
fora 
leave 


ES691 - Mathematics for Machine Learning / Dr. Naveed R. Butt @ GIKI - FES 51 


To Murree or Not to Murree... 


If input > 0.3 
apply for 
| leave 
Else 
Activation ignore 


Let’s go 
to 
Murree Apply 

for leave? 


I’ve 
applied 
fora 
leave 


ES691 - Mathematics for Machine Learning / Dr. Naveed R. Butt @ GIKI - FES 52 


Of course, you could replicate these 
Simple components and simple rules... 


Of course, you could replicate these 
Simple components and sim”!> "les... 


apply for 
leave 
Else 
Activation ignore 


Apply 
for leave? 
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Btw, goes this remind you OT 
something? 


Let’s go 
to 
Murree 


I've 
applied 
fora 
leave 


0.1 
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Apply 
for leave? 


Pas 


Btw, goes this remind you OT 


something? 


Let’s go 
to 
Murree 


I've 
applied 
fora 
leave 


0.1 


Apply 
for leave? 


Breaking a 
problem into 


simpler units and 
rules! 
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You may also want to do it in 
reverse... 


You may also want to do it In 


reverse... 


Did Taj say 
“Let's go 
to Murree”? 


How can we 
use this 
information 
to set 


weights and 
Did Ali say threshold? 
“Let's go 
to Murree”? 
Did we 
end up 
: going to 
se pay Murree? 
applied for “Learning” SIMPLE 
a leave”? RULES for SIMPLE 
. REPEATED UNITS to 
nat All say help solve difficult 
ve 
applied for ES691 - Mathematics for Machine Learning / Dr. Naveed R. Bt problem. 


a laave”? 


“Learning” rules from input-output or 
history... 
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“Learning” rules from input-output or 
history... 
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“Learning” rules from input-output or 
history... 
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“Learning” rules from input-output or 
history... 
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“Learning” rules from input-output or 
history... 


011235813 2134... 


10 20 
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“Learning” rules from input-output or 
history... 


Sas) «6 «CSS 


ee 6 hU 
Oa = il 


10 20 
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Learning iteratively VS. VIFeCct 
Solution 


Showing that iterative 


learning with simple 
rules can work! 


Suppose you have to learn a and b 
iteratively with following rules 


- Weinitializea =1,b=1 

- After each iteration, either a or 
b can go up or down by 1, but 
not both. 

- Update criteria is to minimize 
e = |Zq — z|, where Zg= desired 
value, and z = obtained value. 
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Learning iteratively VS. VIFeCct 
Solution 


Showing that iterative 


learning with simple 
rules can work! 


Suppose you have to learn a and b 


a | | ee | ees 


We initializea =1,b=1 

- After each iteration, either a or 
b can go up or down by 1, but 
not both. 

- Update criteria is to minimize 
e = |Zq — z|, where Zg= desired 
value, and z = obtained value. 
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Learning iteratively VS. VIFeCct 
Solution 


Showing that iterative 


learning with simple 
rules can work! 


Suppose you have to learn a and b 


a ees 


We initializea =1,b=1 

- After each iteration, either a or 
b can go up or down by 1, but 
not both. 

- Update criteria is to minimize 
e = |Zq — z|, where Zg= desired 
value, and z = obtained value. 
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Learning iteratively VS. VIFeCct 
Solution 


Showing that iterative 


learning with simple 
rules can work! 


Suppose you have to learn a and b 


ee | | ee | ees 


We initializea =1,b=1 

- After each iteration, either a or 
b can go up or down by 1, but 
not both. 

- Update criteria is to minimize 
e = |Zq — z|, where Zg= desired 
value, and z = obtained value. 
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Learning iteratively VS. VIFeCct 
Solution 


Showing that iterative 


learning with simple 
rules can work! 


Suppose you have to learn a and b 


ee | | ee | ees 


We initializea =1,b=1 

- After each iteration, either a or 
b can go up or down by 1, but 
not both. 

- Update criteria is to minimize 
e = |Zq — z|, where Zg= desired 
value, and z = obtained value. 
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Learning iteratively VS. VIFeCct 
Solution 


Can we do 
~ something 


Showing that iterative 


learning with simple 
smarter rules can work! 


ba bout this 


Suppose you have to learn a and b 
iteratively with following rules 


We initializea =1,b=1 

- After each iteration, either a or 
b can go up or down by 1, but 
not both. 

- Update criteria is to minimize 
e = |Zq — z|, where Zg= desired 
value, and z = obtained value. 
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Learning iteratively VS. VIFeCct 
Solution 


But we could have 
done this directly as 


well... 


Yes, but remember 

- We are showing that it is possible 
iteratively with simple rules 

- As the problem becomes bigger and 
more intractable, direct solutions will 
be elusive, while the simple iterative 
rules could still work with lots of nodes 
with simple rules. 

- In fact, in many real world problems we 
may not even be sure how to define 
parameters of a problem (but such 
repeated simple structures may co 


me 
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Learmming iteratively VS. VIFECt 


Solution 


Z=ax+by 


Yes, but remember 

- We are showing that it is possible 
iteratively with simple rules 

- As the problem becomes bigger and 
more intractable, direct solutions will 
be elusive, while the simple iterative 
rules could still work with lots of nodes 
with simple rules. 


- In fact, in many real world problems we 


may not even be sure how to define 
parameters of a problem (but such 


repeated simple structures may come | 
E5091 - Mathematics tor 


in handy). 


But we could have 


done this directly as 
well... 


Is This a | 
Good 
Poem for | 
the 
Course? 


i 
| 


ee AaArninn | Mr Navan D PRiitt Mm Ci! e 2 
aa ( > Y 2 | a) Gil 
AM It Wi | i [ \ \ 
~ N 


OK, but do these kind of learning approaches 
actually lead to something amazing? 


OK, but do these kind of learning approaches 
actually lead to something amazing? 


(Really?) 


NIH RESEARCH MATTERS 


= Personalized 
AI tool predicts response to cancer therapy MI e d | C l ne 


At a Glance 


e Scientists developed an AI tool that uses routine clinical data to identify cancer patients most likely to respond to 
immunotherapy drugs called checkpoint inhibitors. 
¢ The approach could help guide personalized cancer treatments for patients. 


Tumor cells 


©@2® °.< 
woe 3 


Predictions of drug response 


ENGI 


DEVELOF 


Single-cell RNA Artificial 
intelligence 


sequencing data 


Generative VS Predictive Al 


fororol 
ol 10 
i} ' 


t) 0 
fororol 


Creativity vs 
Insight 
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Generative VS Predictive Al 


Ibné Muneeb a 11010! 
@ August 17 at 11:07PM -@ ; & 
Asked ChatGPT to explain one of my ghazal's. Quite impressive 
response (English translation of the verses is also by ChatGPT) 


Certainly! Here’s an explanation of the poem by **Ibne Muneeb**: 


### **Explanation of the Poem** Creativity VS 
93 ug) healé gue ugl> 1” iene ay las jl Sil 4 
— ligy ylgsb> paw — big yi > S> 
365 9> 9 i id YL mia IS 3)> 93 wigS 
** ; ligy Su 395 09 **1 y Gigy ylawl yay 
**When there are distances in hearts, **Unique melodies play 
Travel becomes arduous. When the liver becomes a string. 
A home that does not offer shelter, Any tale of pain 
Is ahome that is of no use.** Is hidden behind verses.** 
- **Explanation**: The poet starts by expressing that emotional - **Explanation**: The poet uses a metaphor of the liver (often 
distance between people makes any journey or relationship difficult. A associated with deep emotions in classical literature) being used as a 
home that does not provide warmth and safety is rendered useless. musical string to describe how deep emotions create unique and 
This highlights the importance of emotional connection and genuine profound expressions. Stories of pain are often concealed within 
support in relationships. poetic verses, illustrating how art can express what is felt internally. 
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Urdu y¥ << English v 


Un Ai Sle asi Strange instruments 
ea Be Jb = > play 
ae dad, : P 
) a ake When the liver is 
a ee es wired 
A story of pain 
; icy jLu —eSgil .4** 
So there is poetry! aaa a 
Enea et ligy )U > So 
4) o ®® G m2 IS dd 95 wig 
**1 4 Uigy ylal Gay 
Open in Google Translate + Feedback 
**Unique melodies play 
When the liver becomes a string. 
Any tale of pain 
Is hidden behind verses.** 
- **Explanation**: The poet uses a metaphor of the liver (often 
associated with deep emotions in classical literature) being used as a 
musical string to describe how deep emotions create unique and 
profound expressions. Stories of pain are often concealed within 
poetic verses, illustrating how art can express what is felt internally. 
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Naveed R. Butt @ GIKI - FES 


BI BIC! 


Home News Sport Business Innovation Culture Travel Earth Video Live 


How Google's balloons surprised their 
creator 
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It was only later that they realised what was happening. Unexpectedly, the artificial 
intelligence (AI) on board the balloon had learned to recreate an ancient sailing 
technique first developed by humans centuries, if not thousands of years, ago. 
"Tacking" involves steering a vessel into the wind and then angling outward again so 


that progress in a zig-zag, roughly in the desired direction, can still be made. 


Under unfavourable weather conditions, the self-flying balloons had learned to tack 
all by themselves. The fact they had done this, unprompted, surprised everyone, not 
least the researchers working on the project. 


Tacking 
Zo tack 
No-sail zone 
? 


? 
¢ 
¢ 
? 
Starboard tack 


s 


® 
Port tack 
¢ 
¢ 
¢ 
¢ 

Starboard =\ 

© ca 
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The Surprising Creativity of Digital Evolution: A Collection of Anecdotes 
from the Evolutionary Computation and Artificial Life Research Communities 


Joel Lehman", Jeff Clune’ *', Dusan Misevic*!, Christoph Adami*, Lee Altenberg®, Julie Beaulieu®, Peter 
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om 


The Surprising Creativity of Digital Evolution: A Collection of Anecdotes 
from the Evolutionary Computation and Artificial Life Research Communities 


Joel Lehman"', Jeff Clune? 2, Dusan Misevic*!, Christoph Adami*, Lee Altenberg®, Julie Beaulieu®, Peter 


Some behaviors 


are quite creative! 
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Ul 
N 


BIW...Despite All the Learning Power and Training, 
System Could fail... 


+ .007 x 


“oanda’” noise “gibbon” 


57.7% confidence 99.43% confidence 


ES691 - Mathematics for Machine Learning / Dr. 


Naveed R. Butt @ GIKI - FES 22 


The Magic of Learning! 


¢How do you think machines learn? 
° Mostly algorithmically! 
els there any other way? 


The Magic of Learning! (In Smart 
Materials...) 


] 


FOLD SMA WIRE ROUND STEEL JIG ? 
* AND PASS CURRENT THROUGH IT ‘ 


CHANGE SHAPE OF SMA WIRE 


SMA WIRE 
\ 
\ 
STEEL JIG 


BATTERY 


- Trained using 


some physical 
property. 

BATTERY CONNECTOR 

PLACE IN HOT WATER 


- Returns to 
4 SMA RETURNS TO 

* PROGRAMMED SHAPE 
ES 


trained state 


under “right” 
\ | 


environment. 
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The Magic of Learning! (In Smart 
Materials...) 


- Trained using 
some physical 
property. 


- Returns to 
trained state 
under “right” 
environment. 
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Questions?? Thoughts? ? 


5 


e 


”-~ — 
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